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ABSTRACT 

This paper describes a needs assessment process that 
identifies a range of performance improvement strategies using the 
steps of the Evaluation Research Model (Geroy and Wright, 1988). A 
detailed description of a project conducted by the Professional 
Development Committee of a mid-sized community college in suburban 
Philadelphia to identify professional development activities for the 
administrators , faculty, and support personnel of the college is used 
to illustrate this process. All of the challenges identified by 
outside consultants were addressed: (1) all employees (stakeholders) 
were involved equally in identifying professional development needs? 
(2) the planning committee recognized that certain non-training 
actions were required to maximize the worth of training? (3) data 
were elicited from all 280 employees that represented both their 
wants and their needs? and (4) decision makers were provided with a 
systematic process for selecting professional development activities 
that will yield maximum results for cost and effort. The project 
illustrated the number and kind of responses generated through the 
nominal group technique (NGT) , as well as the range of professional 
development options that emerge from NGT responses. It reflected the 
cutting edge of performance technology by virtue of its application 
in an academic setting, involving all employees of the organization, 
addressing the desire of the organization for training while 
simultaneously presenting alternative performance improvement 
solutions, and introducing performance technology as the decision 
making model for all HRD (human resource development) functions of 
the organization. The project also provided techniques, references, 
and examples that can be used in convincing decision makers of the 
value of conducting a needs assessment. (5 references) (BBM) 
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Introduction 

The effectiveness of any organization is dependent on its most important resources: 
people. Systems and techniques abound for the purpose of developing human resources. 
These are designed to help an organization get the best performance from some type of 
training. Analysis can show what kind of training is most appropriate for achieving improved 
performance and also reveal when training is not necessary in order to achieve improved 
performance. One method that accomplishes both of these ends is evaluation research (Qeroy 
and Wright, 1988). 

Figure 1 
Evaluation Research Model 
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Qeroy and Wright, 198B 



Evaluation research is a pragmatic, program focusod research strategy of analysis for 
decision makers. Its purpose is to provide data that can enable decision making, through 
analysir of pros and cons, through prioritizing, or through the application of decision making 
criteria, etc. What gives evaluation research its strength is the extent to which the stakeholders 
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are involved throughout the research process. This involvement yields four primary benefits: 
feasibility in terms of cost, timeliness and manner of implementation; and utility , accuracy , and 
proprietorship in terms of outcome. 

Beginning with Phase One: Selection of Evaluation Research Purpose (See Figure 1), 
the research evaluators and the stakeholders "become collaborating partners in the search for 
useful information M (Geroy and Wright, 1988, p. 23). From then on, collaboration is structured at 
each step of the process. This collaboration provides a means of monitoring the 
appropriateness and relevancy of each activity and strategy, as well as its feasibility regarding 
time and resources. As a result, the information gathered is more likely to be on target in 
addressing the needs of the stakeholders. It is also more likely to facilitate decision making both 
through its accuracy and the usefulness of the way the information is reported. 

Furthermore, those who have helped structure the process, as well as those from whom 
the data is derived, have greater ownership of the results. Their potential buy-in to the 
recommendnthns stemming from the research is Increased (proprietorship). In each case the 
data gathered are used to either create, maintain or improve program policy and / or 
implementation practices. 

This article describes a needs assessment process that identifies a range of 
performance improvement strategies using the steps of the Evaluation Research Model. 

Overview 

The Professional Development Committee of a mid-sized community college in 
suburban Philadelphia had the responsibility of identifying professional development activities 
for the administrators, faculty and support personnel of the college. In light of concern voiced 
over past professional development offerings, the committee decided to use outside 
consultants to help them plan their professional development program. After three lengthy 
meetings with the committee, the consultants identified five problems imbedded in the 
environment that were influencing the decision making process: 

• lack of agreement among committee members as to how to select professional development 
opportunities, 

• limited communication of information across functional groups, 

• negative perceptions of previous professional development efforts, 

- perceived unresponsiveness of decision makers to past suggestions related to professional 
development, and 

• recent loss of incentive due to reduced power and autonomy within the faculty structure. 

It became apparent that the final professional development plan for the 1991-92 
academic year had to: 

• involve and empower all employees (stakeholders) equally in identifying professional 
development needs, 

• develop a means by which the planning committee itself would recognize that certain non- 
training actions were required in order to maximize the worth of training, 

• elicit data that would represent needs of the employees of the college, 

• elicit valid data while facilitating a process with 280 participants, and 

• provide decision makers with systematic process for selecting professional development 
activities that will yield maximum results for cost and effort. 

Using the evaluation research strategy (Geroy & Wright, 1988) as a guideline, a needs 
assessment was conducted with 280 support-staff, faculty and administrators. The Nominal 
Group Technique (Debequa & Van Deben, 197^), a comprehensive process for data gathering 
within a group setting, was taught to representatives from each of these three groups. These 
representatives facilitated the participation of virtually all employees at the college in identifying a 
prioritized list of items that would enable these persons to "do their job better" - from their 
perspective. _Analysis of the suggested performance improvement needs utilized the 
Performance lechnology Model (Gilbert, 1978) and revealed that the solutions fell into five 
categories of intervention: high yleid training, direction and flow of information, resources, 
performance incentives, and medium yield training. Results indicated that improving employee 
performance required aftering information, resource and incentive systems and that without 
these alterations, most training was likely to have minimal effect. 
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Methodology 

Each btage of the Evaluation Research Model was incorporated into some phase of the 
project. In clarifying the goals of the professional development planning process, the 
Professional Development Committee and the consultants used group discussion and the 
Nominal Group Technique as they: 

• Selected and Evaluated the Research Purpose: Needs Assessment - Stage One, 

• Identified the Technique to Be Used: Front End Analysis - Stage Two, and 

• Developed the Research Question: "What do I need to do my job better?" - Stage Three. 

Focus groups were then held with representatives from each functional area in order to 
identify any perceptions or problems in the workplace that might have an impact on the planning 
process. At this point, the Committee and the consultants: 
• . otablished a Collaborative Utilization Focused Process - Stage Four, and 

• Determined Research Focus: Evaluation Research Without Goals - Stage Five. 

With the data gathered through committee meetings and the focus groups, the 
consultants then: 

• Determined the Evaluation Research Strategy: Anthropological-Holistic-Inductive - Stage Six 

• Selected the Data Collection Method: Non-formal Instrument, Nominal Group Technique - 
Stage Seven 

Assessment Technique 

The Nominal Gioup Technique was used to gather data for the needs assessment 
(Debeque and Van Deben, 1974). This technique utiles a structured group meeting 
conducted by a group leader or facilitator in five steps. 
STEP ONE: 

• Group members sit around a table, but initially, no talking takes place. Each individual has a 
sheet of paper with the "nominal question" on it. This question provides the primary focus of 
the meeting. This question is carefully constructed prior to the meeting in order to generate 
the required information. Participants, independently and silently, write down as many 
answers to the question as possible. 

STEP TWO: 

• After approximately ten minutes, the facilitator , going round-robin, calls on each member of 
the group to give one of his or her ideas. Each idea is listed on a flip chart and numbered 
sequentially. The purpose of this stage is to make sure that each participant is given equal 
opportunity to share his or her ideas, so that highly verbal individuals are not dominant. Thus 
all discussion and judgement are postponed. 

STEP THREE: 

• The facilitator reviews each idea sequentially, encouraging clarification questions, 
elaborations, support for an idea, as well as rebuttals, or hitch-hiking to new ideas. This 
phase is complete when all ideas have been reviewed. 

STEP FOUR: 

• This stage is optional depending on the number and kind of responses generated in Step 
Two. When there is great overlap of ideas, the group can categorize ideas by topic. This can 
facilitate the ranking that follows in Step Five. It is necessary, however, for members of the 
group agree on the categories established and the items that go into them. 

STEP FIVE: 

• Each participant silently and privately ranks the ideas by assigning a numerical value to the 
id?a. Depending on the purpose of the NGT the ranking criteria couid be cost of 
implementation, feasibility, importance, etc. The highest number in the ranking being the 
total number of the ideas (or categories). Each member's ranking of each idea is recorded 
and the average rank of each item is derived. This yields the group's priorities in relation to 
the ideas. 

The advantages of the NGT are numerous, especially when one of the goals of the 
activity is to involve stakeholders. The NGT assures that each person has an equal opportunity 
to express his or her ideas and protects against dominant personalities. It stimulates the 
generation of ideas through silent writing in Step One and the round-robin listing in Step Two, 
thus preventing closure on ideas before all are equally considered. All participants have an 
opportunity to reflect on all ideas and have their questions and concerns addressed. Silent 
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ranking gives equal weight to each opinion during decision making and reduces peer pressure 
to support one idea. Furthermore, the NGT is cost and time effective. (Scott and Deadrick, 
1982) 

Implementation, Stage Eight of the Evaluation Research Model, took place in several 
steps. Representatives from each functional group were identified as facilitators of the NGT. 
These volunteers were trained for one-half day in how to facilitate the NGT. Finally, all employees 
of the college were gathered for a day and a half during which three rounds of the NGT were 
conducted. 

On the morning that all employees meet together, the Needs Assessment Process was 
introduced and ice breaker activities, focusing on the mission of the college, were conducted. 
These activities were designed to encourage the free flow of ideas during the afternoon NGT 
sessions. 

After lunch, each functional group, administrators, support staff and faculty, met in a 
different location. There they divided into groups of ten where the trained volunteers facilitated 
NGT Round One: Identify Needs and Round Two: Prioritize Needs. The question upon which 
the NGT was focused was "What do I need to do my job better?" All data from both Rounds were 
collected and volunteers from the support staff, some members of the Professional 
Development Committee members and the consultants worked until 10:00 pm organizing, 
typing and copying ALL responses. 

The next morning each of the 280 participants received a copy of the responses of his or 
her functional group, as well as the responses of the other two groups. This served several 
purposes. It facilitated Round Three: Generate Solutions, by increasing the likelihood of 
constructive solutions. For instance, a review of all of the responses might reveal that some of 
the needs that might have appeared valid at first glance, were less so when compared to needs 
from other groups. The quick turn around also demonstrated responsiveness on the part of the 
committee to employee suggestions. 

By the end of the day and a half, the project had evolved through the first eight stages of 
the Evaluation Research Model and utilized the Nominal Group Technique in three separate 
contexts: establishing the goals of the project (15 participants), training the facilitators (40 
participants), and conducting the needs assessment (280 participants). 

Stage Nine: Analyzing and Reporting Results was carried out over the six weeks that 
followed. During this time, The Professional Development Committee categorized all of the 
needs and solutions using the Performance Technology Model. The resulting categories 
facilitated the decision making analysis. 

Analysis Model 

The premise of performance technology is that there are a variety of factors in the 
workplace that have bearing on performance effectiveness and that appropriately adjusting 
these factors can yield exemplary performance, often without training. Thus, it behoves those 
responsible for the performance of others to identify environmental and individual factors in a 
given workplace that will not only improve performance, but also maintain exemplar quality 
without costly training. 

These factors are perhaps best represented by the Behavior Engineering/PROBE 
Model developed by Thomas Gilbert (1978, 1982) which is the basis of the Performance 
Technology Model (See Figure 2.). This model identifies the kinds of things that can be done to 
improve and maintain performance, one of the primary goals of any professional development 
project. 

It is Gilbert's claim that any job that is supported in all six of ihe areas of the Model should 
"carry a guarantee cf high competence, provided that management was structured so as to really 
deHver these things and had a clear focus on the mission of the job in the first place" (1978, p. 

Research and experience verify that without good needs assessment, training is likely to 
be the performance improvement intervention of choice. When training is used, but it's not the 
most effective solution; wants instead of needs are usually being addressed. At the same time, 
the value of training yields less than optimal return on the dollar and time investment, and has 
little long term impact on the organization as a whole (Kaufman, 1986). 

Strategies related to evaluation research and performance technology have proven 
effective in both carrying out "front end" identification of needs (Geroy & Wright, 1988; Gray 
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1987; Debeque & Van Deben, 1974 ) and deciding how to address the needs so that the 
training that is eventually done will have high yield to the trainee and the organization (Earle, 
1990). Performance technology helps to indicate where training is no: the best choice of 
intervention and if training is the best intervention, what kind of training is best suited to meet the 
need. 

Figure 2 
Behavior Engineering Model 
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(Gilbert, 1978) 

Results 

The solutions generated from Round Three of the Nominal Group Technique were 
tabulated by the volunteers over a six week period. Lists of prioritized needs and suggestions 
were then organized by kind in order to facilitate analysis. For instance, all suggestions that had 
to do with the need for clarification or revision of the budget were grouped together. This is also 
a category of solutions. The process of grouping was conducted separately for faculty, support 
staff and administrators. 

At the conclusion of this process, three separate lists of needs/solutions were forwarded 
to the consultants. Review of the data indicated that the groups of ideas/needs could be 
combined into at least one of five categories, depending on the type of solutions that would 
meet the need. 

Category One: High Yield Development Programs 

• The needs in this category could be met by effectively designed training that matched the skill 
and knowledge requirements of a specific job function. The term high-yield indicated that the 
benefit from the program was greater than the cost of implementing the program. 

Category Two: Information 

• Three general solutions met the needs in this category. The first solution was to provide 
descriptions of what was expected of performance. Second, to develop clear and relevant 
guides for job performance. Third, to provide relevant and frequent feedback about the level of 
performance relative to the expectations. 
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Figure 3 
Findings: Administration 
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The counts in the cells represent '.he number of times a need was indicated. 



Category Three: Resources 

• This category involved needs that could be solved by more equipment, better tools, a more 
effective working environment, or other solutions that require additional funding. 

Category Four: Incentives 

• Needs that could be met by adequate financial and non-financial incentives fall into this 
category. Optimally, these incentives should be based on performance. Also, career- 
development opportunity solutions addressed these needs. 

Category Five: Moderate-Yield Development Prngrarm 

• This category was similar to Category One except for the yield derived from the implementation 
of the program. The cost of the program would not have been balanced by the benefit. This was 
probably due to another related need that fell into the Second, Third or Fourth Categories. It was 
recommended that programs in this category not be implemented until the other related needs 
were met. Once the related needs were met, however, the yield of the program might change, 
or the need for the program might be eliminated entirely. 

These categories represent four of the cells of the Behavior Engineering Model. 
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Figure 4 
Findings: Support 
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In comparing the categories across groups, it is interesting that, in each case, information 
needs/solutions were offered more than any other, and that resource needs/solutions were 
second. Thss is consistent with the experience of many performance technologists who often 
find, during the course of needs assessments, that information, in the form of clearly defined 
performance expectations, policies, procedures and feedback, is the greatest opportunity for 
improvement related to performance improvement. (Gilbert, 1991) 

Providing information is less time consuminr and less costly than providing training. It is 
also more likely to result in performance improvement. Training often occurs when an 
information problem still exists. Employees wiil still not be clear on performance expectations 
and do not receive feedback that wiil help them assess their progress towards performance 
expectations. In these situations, they simply cannot make maximum performance improvement. 
This results in frustrated employees and a frustrated organization. 

As a result of the project findings and recommendations, the Professional Development 
Committee undertook the following follow-up activities. 

• Conduct re-orientation program for a!! employees (INFORMATION) 

• Use re-orientation program as the new employee orientation program (INFORMATIONS 

• Implement training identified as high yield: (TRAINING) 

- Interpersonal Problem Solving for administrators 

- Communication Skills for faculty and support staff 

- PC Teaching Methods for faculty 

- Delegation Skills for administration 

- Budget Planning for administration 
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Figure 5 
Findings: Faculty 
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• Establish a new committee to oversee non-instructional performance improvement solutions 

• Present an overview of performance technology to the Professionai Development Committee 

• Review results of training and non-instructional interventions after one year 

The solutions reported in the resources, incentives and information categories were not 
immediately addressed by the committee for two reasons. Their mission had been to plan 
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professional development activities within the traditional paradigm of professional development. 
Once their vision of professional development had been expanded to include non-training 
factors that support performance improvement, they felt that the most responsible strategy 
would be to have a new committee established that could focus exclusively on non-instructional 
performance improvement solutions. Secondly, the original committee was not empowered to 
authorize the budget expenditures that might be necessary to implement resource and 
incentive solutions. 

Conclusions 

The benefits from this project are numerous. Evaluation research (Geroy & Wright. 
1988) facilitated the decision making regarding the relative worth of and priority of actions to 
promote both professional growth and institutional development. Without the project, the 
Professional Development Committee would have selected training activities that might, or might 
not, have addressed some of the needs of the college employees. Certainly, they would not 
have considered non-instructional needs. 

All of the challenges that were identified by the consultants at the beginning of the 
project were addressed: 

• All employees (stakeholders) were involved equally in identifying professional development 
needs. This was particularly significant for the support staff, whose work keeps an organization 
moving. It was the first time they felt anyone had listened to their concerns or ideas. 

• The planning committee did recognize that certain non-training actions were required in order 
to maximize the worth of training 

• Data was elicited from all 280 employees that represented both wants and needs of the 
employees of the college. The degree of overlap of needs/solutions validated both the data and 
the effectiveness of the process, 

• Decision makers were provided with with systematic process for selecting professional 
development activities that will yield maximum results for cost and effort. 

There were also benefits related to the practice of performance technology. This project 
illustrated the number and kind of responses generated through the nominal group technique, 
as well as the range of professional development options that emerge from NGT responses. It 
reflected the cutting edge of performance technology by virtue of: applying of performance 
technology in an academic setting, involving all employees of the organization, addressing the 
desire of the organization for training while simultaneously, presenting alternative performance 
improvement solutions, and introducing performance technology as the decision making model 
for all HRD functions of the organization. The project also provided techniques, references and 
examples that can be used in convincing decision makers of the value of conducting a needs 
assessment. 
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